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ABSTRACT 


The  End-To-End  (ETE)  Test,  conducted  under  the  auspices  of  the  Department  of  Defense  Joint  Advanced 
Distributed  Simulation  (JADS)  Joint  Test  and  Evaluation  (JT&E),  is  currently  developing  a  synthetic  test 
environment  that  can  be  used  for  future  operational  testing  and  doctrinal  development.  This  synthetic  test 
environment  will  be  used  initially  to  conduct  developmental  and  operational  testing  of  the  Joint 
Surveillance  Target  Attack  Radar  System  (Joint  STARS).  As  designed  and  built,  it  may  be  used  to 
conduct  future  testing  of  systems  such  as  the  Common  Ground  Station  (CGS),  the  All  Source  Analysis 
System  (ASAS),  and  the  Block  II  Army  Tactical  Missile  System. 

This  paper  will  describe  progress  to  date,  to  include  the  results  of  our  initial  operational  and 
developmental  tests,  lessons  learned  to  date,  and  experience  gained  in  verifying  and  validating  a  synthetic 
test  environment  using  the  Department  of  Defense  Verification,  Validation  and  Accreditation  (VV&A) 
Recommended  Practices  Guide. 

Background 


The  Joint  Advanced  Distributed  Simulation  (JADS)  Joint  Test  and  Evaluation  (JT&E)  was  chartered  by 
the  Office  of  the  Under  Secretary  of  Defense  (Acquisition  and  Technology),  Office  of  the  Director,  Test 
and  Evaluation  “...to  investigate  the  utility  of  Advanced  Distributed  Simulation  (ADS)  for  both 
developmental  test  and  evaluation  (DT&E)  and  operational  test  and  evaluation  (OT&E).  JADS  will 
investigate  the  present  utility  of  ADS,  including  Distributed  Interactive  Simulation,  for  test  and  evaluation 
(T&E);  and  finally,  identify  the  requirements  that  must  be  introduced  in  ADS  systems  if  they  are  to 
support  a  more  complete  T&E  capability  in  the  future.” 

The  End-To-End  (ETE)  Test  is  one  of  the  three  tests  within  the  JADS  JT&E  program.  The  ETE  Test  is 
designed  to  evaluate  the  utility  of  ADS  to  support  the  testing  of  a  C4ISR  system,  the  Joint  Surveillance 
Target  Attack  Radar  System  (Joint  STARS),  while  the  system  performs  the  end-to-end  loop  of  detect, 
track,  target,  cue  a  weapons  system,  and  assess  battle  damage. 

Joint  STARS  is  composed  of  both  an  airborne  and  ground  segment  along  with  the  necessary 
communications  subsystems.  The  E-8  airborne  system  and  the  Ground  Station  Modules  (GSM)  together 
provide  the  surveillance,  target  detection,  and  tracking  required  to  assist  commanders  in  understanding 
the  enemy  situation  and  taking  action  to  destroy  enemy  forces. 

To  support  the  ETE  Test,  a  simulation  called  the  Virtual  Surveillance  Target  Attack  Radar  Sensor 
(VSTARS)  was  designed  by  JADS  and  built  by  Northrop  Grumman,  Lockheed  Martin,  and  Motorola. 
VSTARS  is  capable  of  functioning  in  a  pure  laboratory  environment  using  commercial  off  the  shelf 
(COTS)  equipment,  in  a  hardware  in  the  loop  environment  using  fielded  components  (Primary  Mission 
Equipment  (PME)),  and  onboard  the  aircraft  coexisting  with  the  real  radar  subsystem  in  an  ADS 
augmented  environment. 


The  ETE  Test  uses  some  of  the  developmental  and  operational  test  issues  for  the  Joint  Surveillance  Target 
Attack  Radar  System  (Joint  STARS)  to  conduct  its  T&E  utility  evaluation  in  an  ADS-enhanced  test 
environment.  Additionally,  the  ETE  Test  will  also  conduct  an  evaluation  of  the  JADS  Test  Control  and 
Analysis  Center’s  (TCAC)  capability  to  control  a  distributed  test  of  this  type  and  remotely  monitor  and 
analyze  test  results. 

The  ETE  Test  consists  of  four  phases.  Phase  1  developed  VSTARS  and  modified  or  developed  the 
components  required  to  establish  the  synthetic  environment  (SE).  A  copy  of  the  Phase  1  final  report  is 
available  on  the  JADS  web  site.  Phase  2  conducted  the  VV&A  of  the  SE  and  evaluated  the  utility  of  ADS 
to  support  DT&E  and  early  OT&E  of  a  C4ISR  system  in  a  laboratory  environment.  A  copy  of  the  Phase  2 
final  report  will  be  on  the  JADS  web  site  in  the  near  future.  Phase  3  will  transition  the  radar  emulation 
portion  of  VSTARS  to  the  E-8C  and  repeat  the  VV&A.  Phase  4  evaluates  the  ability  to  perform  OT&E  of 
Joint  STARS  in  a  synthetically  enhanced  operational  environment. 

As  stated  in  the  abstract,  this  paper  will  present  some  of  the  results  of  our  initial  operational  and 
developmental  tests,  lessons  learned  to  date,  and  experience  gained  in  verifying  and  validating  a  synthetic 
test  environment  using  the  Department  of  Defense  Verification.  Validation  and  Accreditation  ('VV&A') 
Recommended  Practices  Guide. 

Verification,  Validation  and  Accreditation  (VV&A) 

One  of  the  major  issues  associated  with  the  use  of  Advanced  Distributed  Simulation  (ADS)  is  the 
credibility  of  the  models  and  simulations  (M&S)  used  and  the  credibility  of  their  interactions  over  a 
distributed  network.  The  credibility  of  M&S  is  commonly  measured  by  verification  and  validation  (V&V) 
and  then  formally  approved  as  adequate  for  use  for  a  specific  application  by  accreditation.  The  V&V  of 
M&S  and  distributed  simulations  is  formally  required  by  Department  of  Defense  (DoD)  Instruction 
5000.61  prior  to  their  use  in  testing. 

The  basis  for  the  VV&A  of  the  Phase  2  SE  was  the  Department  of  Defense  Verification,  Validation  and 
Accreditation  (VV&A')  Recommended  Practices  Guide  and,  since  the  ETE  Test  utilizes  the  Institute  of 
Electrical  and  Electronics  Engineers  (IEEE)  Standard  1278  for  Distributed  Interactive  Simulation  (DIS) 
to  develop  its  ADS  SE,  use  was  also  made  of  the  Recommended  Practice  for  Distributed  Interactive 
Simulation  —  Verification,  Validation,  and  Accreditation  (Draft-4  November  1996). 

The  V&V  of  the  components  of  the  SE  and  the  SE  itself  was  focused  upon  the  functional  requirements 
and  acceptability  criteria  from  the  test  plan  and  other  documents  that  describes  the  SE.  This  allowed  the 
V&V  agent  to  focus  his  efforts  on  those  requirements  critical  to  the  conduct  of  the  ETE  Test  and  enabled 
him  to  develop  the  V&V  plan  by  identifying  the  tasks  required  to  meet  these  requirements  and 
acceptability  criteria.  The  V&V  plan  was  developed  in  a  manner  that  matched  and  complemented  the  test 
activity  plan,  test  requirements,  component  requirements,  available  resources,  and  timelines  for  the  test. 

The  results  of  the  V&V  were  presented  to  an  accreditation  board  composed  of  DoD  and  Multi-Service 
representatives  and  the  ETE  Test  SE  was  accredited  for  use  in  the  Phase  2  OT  Test  by  the  Director  of 
JADS,  Col  Mark  Smith.  Associated  plans,  results,  and  briefings  may  be  accessed  on  the  JADS  web  site. 

W&A  Lessons  Learned 


With  the  publication  of  the  Department  of  Defense  Verification,  Validation  and  Accreditation  (VV&A') 
Recommended  Practices  Guide  and  other  guides,  such  as  the  Recommended  Practice  for  Distributed 
Interactive  Simulation  —  Verification,  Validation,  and  Accreditation  (Draft-4  November  1996J.  there 
exists  a  plethora  of  processes  and  practices.  The  difficulty  lies  in  applying  them  to  your  particular 
program  as  executable  tasks  at  the  appropriate  stages  of  your  test.  The  solution  arrived  at  by  the  ETE 
V&V  team  was  to  join  the  crowd  and  develop  our  own  JADS  ETE  Test  VV&A  Process  Model. 


Figure  1.  JADS  ETE  Test  W&A  Process  Model 

In  the  JADS  ETE  Test  VV&A  Process  Model,  test  events,  which  consist  of  the  planning,  construction  and 
assembling  of  the  SE,  integration  and  testing  of  the  SE,  accreditation  of  the  SE,  and  conduct  of  the  test  all 
proceed  on  the  left  side  from  top  to  bottom.  The  V&V  events,  to  include  documentation,  proceed  to  the 
right  for  each  test  event. 


We  also  learned,  or  relearned,  that  V&V  cannot  be  economically  and  efficiently  performed  without  a  set 
of  documented  requirements  and  acceptance  criteria.  We  learned  this  the  hard  way  because  up  to  the 
point  of  writing  the  V&V  plan,  we  had  never  sat  and  written  down  the  requirements  and  acceptability 
criteria  for  the  SE  and  its  components.  They  existed  as  requirements,  but  were  scattered  in  various  plans, 
documents,  and  statements  of  work.  More  importantly,  what  was  acceptable  did  not  exist  in  most  cases 
and  needed  to  be  developed  before  V&V  could  begin.  It  is  one  thing  to  say  that  you  require  the  simulation 
to  simulate  the  radar  onboard  the  E-8C.  It  is  much  more  difficult  to  say  what  is  acceptable.  Acceptability 
criteria  are  the  specifications  the  modeler  must  meet.  Likewise,  these  criteria  must  be  known  before  V&V 
begins  in  order  to  scope  the  cost  and  level  of  effort  of  the  V&V. 


As  an  example,  we  used  an  existing  constructive  simulation  (Janus)  to  provide  Entity  State  Protocol  Data 
Units  (ESPDU)  to  VSTARS.  All  VSTARS  needed  was  what  kind  of  entity  was  it,  where  is  it  at,  and  if 
moving,  how  fast  is  it  moving  and  in  what  direction.  This  meant  that  we  did  not  need  to  verify  and 
validate  the  acquisition  algorithm  in  the  simulation,  nor  did  we  need  to  worry  about  direct  fire 
engagement  outcomes.  As  a  result,  our  V&V  of  this  component  of  the  SE  was  economically  and 
efficiently  performed. 

Finally,  we  learned  that  in  order  for  V&V  to  be  economically  and  efficiently  performed,  it  must  be  done  in 
conjunction  with  the  construction  and  assembly  of  the  SE  and  the  integration  and  testing  of  the  SE.  The 
costs  and  effort  involved  in  activating  the  SE  prohibit  its  activation  for  the  purposes  of  V&V  alone.  In 
addition,  by  doing  V&V  in  this  manner,  V&V  becomes  a  continuous  quality  check  on  the  SE  and  a  tool 
that  continues  to  be  used  even  after  accreditation.  The  results  of  the  Phase  2  OT  were  validated  by 
reviewing  the  data  and  applying  the  V&V  measures  to  ensure  the  SE  behaved  as  expected. 

Developmental  Testing  Lessons  Learned 

VSTARS  was  designed  so  that  the  only  simulation  taking  place  is  the  simulation  of  the  MTI  and  SAR 
radar  modes  within  the  radar  subsystem.  Everything  else  is  either  integration  code  or  actual  E-8C  system 
code.  The  inputs  into  VSTARS,  except  for  the  target  data,  are  normal  inputs  into  the  real  radar  processor 
and  the  outputs  are  the  actual  radar  reports.  The  radar  simulations  are  parallel  processes  with  the  radar 
when  live  and  virtual  are  mixed  and  solve  the  radar  equations  in  order  to  achieve  the  required  fidelity. 

This  architecture  allows  the  use  of  VSTARS  to  conduct  developmental  testing  of  all  of  the  other 
subsystems  that  comprise  Joint  STARS,  provided  that  VSTARS  is  an  accurate  representation  of  the  radar. 
Obviously,  VSTARS  cannot  be  used  to  conduct  developmental  testing  of  the  radar  subsystem. 

As  an  example,  one  of  the  features  of  the  workstation  used  on  the  E-8C  is  an  automatic  tracker  (A- 
Tracker).  The  A-Tracker  works  off  radar  reports  and  when  initiated,  will  automatically  track  a  designated 
formation,  providing  bearing,  speed,  and  number  of  vehicles.  Previous  to  VSTARS,  it  was  necessary  to 
either  have  a  functioning  radar  (test  flight)  or  a  recording  of  a  functioning  radar  in  order  to  test  the  A- 
Tracker.  Test  cases  were  basically  limited  to  those  that  could  be  achieved  at  Eglin  AFB  with  a  minimal 
number  of  vehicles  traveling  under  peacetime  safety  restrictions. 

Northrop  Grumman  is  currently  developing  an  annual  release  of  the  radar  software  that  will  incorporate  a 
revised  version  of  the  A-Tracker  software.  This  provided  JADS  with  the  opportunity  to  ask  Northrop 
Grumman  to  conduct  an  ad  hoc  study,  parallel  to  the  normal  testing  of  the  new  A-Tracker  software,  to 
determine  if  it  would  be  possible  to  use  VSTARS  to  test  this  software. 

Numerous  problems  were  experienced  due  to  the  ad  hoc  nature  of  the  study,  both  in  the  areas  of  software 
integration  and  scenario  generation.  Additionally,  the  V&V  of  VSTARS  had  not  yet  been  completed  and 
thus  no  results  could  be  used  as  documentation  for  the  annual,  release.  Despite  these  problems,  several 
lessons  learned  resulted  from  this  study. 

•  Test  cases  could  be  “flown”  using  VSTARS  whenever  needed  with  as  many  repetitions  as 
desired.  This  was  possible  without  competing  for  scarce  test  aircraft  and  range  resources. 

•  There  was  a  potential  for  enormous  cost  savings.  One  tester  and  two  computers  in  a  lab  vice 
the  aircraft,  testers,  crew,  and  range  assets  required  for  a  live  test. 

•  Any  conceivable  test  case  could  be  “flown”  in  the  laboratory  without  worrying  about  safety 
or  limited  assets,  provided  the  appropriate  scenario  generator  was  available. 

•  Bad  software  could  be  quickly  discarded  and  new  software  could  be  tried  the  next  day. 

•  Most  importantly,  when  a  live  test  flight  is  flown,  as  it  must  be,  the  testers  can  be  reasonably 
sure  that  they  will  get  the  maximum  value  from  the  flight  and  test  conditions. 


Operational  Testing  Lessons  Learned 


The  Phase  2  Operational  Test  conducted  by  the  JADS  ETE  Test  Team  was  designed  to  determine  the 
utility  of  ADS  to  support  early  OT&E  of  a  C4ISR  system  in  a  laboratory  environment.  The  synthetic 
environment  developed  to  support  the  test  is  shown  below  as  Figure  2.  Overview  of  Phase  2  ETE  Test 
Synthetic  Environment. 
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Figure  3.  Overview  of  Phase  2  ETE  Test  Synthetic  Environment 

The  ETE  test  used  a  DIS  compatible  Janus  to  generate  nearly  10,000  entities  representing  the  elements  in 
the  rear  of  a  threat  force.  TRAC-WSMR,  at  White  Sands  Missile  Range,  New  Mexico,  provided  the  Janus 
scenario  feed  to  VSTARS.  The  Test  Control  and  Analysis  Center,  in  Albuquerque,  New  Mexico, 
provided  test  control.  The  JADS  Network  and  Engineering  Team  monitored  the  health  of  the  ETE  test 
network,  and  ensured  that  adequate  data  flowed  in  support  of  the  test.  VSTARS,  as  previously  described, 
provided  the  emulation  of  the  E-8C  from  the  Grumman  Aerospace  Labs. 

The  target  analysis  cell  (TAC),  fire  support  (Advanced  Field  Artillery  Tactical  Data  System,  or  AFATDS) 
and  a  light  ground  station  module  (LGSM)  were  stationed  at  Fort  Hood,  Texas.  Communications  among 
these  C4I  systems  employed  such  doctrinally  correct  means  as  the  CGS-100,  a  subsystem  of  the 
Compartmented  All  Source  Analysis  System  (ASAS)  Message  Processing  System  (CAMPS),  ASAS 
Remote  Workstations  (RWSs),  and  Advanced  Field  Artillery  Tactical  Data  System  (AFATDS)  message 
traffic.  The  Tactical  Army  Fire  Support  Model  (TAFSM)  simulation  modeled  the  Advanced  Tactical 
Missile  System  (ATACMS)  Bn,  and  sent  the  fire  and  detonate  PDUs  to  the  Janus  Vn  6.88D  simulation. 
In  turn,  Janus  modeled  the  engagement  results  and  reflected  the  results  in  the  synthetic  environment. 

Over  40  test  periods  of  more  than  seven  hours  duration  were  conducted  prior  to  the  actual  OT.  During  the 
actual  OT,  five  seven  hour  scenarios  were  ran  and  data  was  collected  that  could  be  used  to  evaluate  Joint 
STARS  MOT&E  measures.  Details  of  the  OT  and  the  results  of  the  evaluation  of  the  value  of  ADS  in 
testing  will  be  available  on  our  web  page  in  the  near  future. 

The  most  significant  finding  as  a  result  of  this  test  was  that  an  ADS  synthetic  environment,  such  as  the 
ETE  Test  SE,  has  significant  utility  in  supporting  early  OT&E  of  a  C4ISR  system  in  a  laboratory 
environment.  Additionally,  this  environment  can  be  used  to  conduct  early  operational  assessments,  and 
for  the  development  of  tactics,  techniques  and  procedures  prior  to  system  testing.  The  SE  also  has 
tremendous  utility  in  test  planning,  rehearsal,  training  of  test  personnel,  verification  of  data  sources  and 
data  reduction  techniques,  and  the  determination  of  whether  adequate  data  is  collected  to  evaluate  the  test 
measures. 

The  use  of  an  ADS  synthetic  environment  also  allows  the  tester  to  conduct  tests  in  an  environment  that  is 
impossible  to  achieve  on  the  test  range.  The  testing  of  a  system  like  Joint  STARS  with  a  brigade  size 


force  is  akin  to  testing  a  five  ton  dump  truck  with  a  bucketful  of  sand.  You  will  be  able  to  tell  if  the  truck 
runs  and  can  carry  sand,  but  you  will  be  hard  pressed  to  extrapolate  your  data  to  make  an  argument  that 
the  truck  can  carry  five  tons.  The  operational  testing  of  large  C4ISR  systems,  such  as  Joint  STARS 
require  a  realistic  operational  environment  to  stress  the  system.  Short  of  arranging  for  a  convenient  war, 
an  ADS  synthetic  environment  appears  to  be  the  only  way  to  achieve  this  test  environment. 

It  was  found  that  the  ETE  environment  may  also  be  used  for  testing  other  systems  that  requires  a  Joint 
Stars  C4ISR  input.  An  example  is  the  Block  2  Army  Tactical  Missile  System  (ATACMS)  that  could  use 
the  ETE  SE  as  is  for  OT&E.  The  only  change  needed  would  be  to  ensure  that  Janus  could  handle  the 
Block  2  weapons  load. 

Finally,  the  use  of  an  ADS  synthetic  environment  for  testing  allows  repeatability.  As  an  example,  let  us 
consider  the  Joint  STARS  operational  measure,  ability  to  detect  an  enemy  convoy.  If  measured  during  a 
live  test,  the  tester  has  no  way  of  knowing  whether  it  is  a  failure  of  the  radar  or  a  failure  of  the  operator. 

If  measured  using  a  SE,  data  can  be  collected  as  to  how  many  repetitions,  if  ever,  it  takes  before  the 
operator  detects  the  convoy  and  the  test  can  be  repeated  with  as  many  operators  as  you  can  find.  Analysis 
of  this  data  should  give  an  indication  as  to  where  the  problem  lies. 

Advanced  Distributed  Simulation  IADS)  Lessons  Learned 


Murphy  knows  how  to  do  ADS  Testing.  As  Hurricane  George  approached  one  of  our  sites  (Melbourne, 
FI),  the  service  providers  decided  to  readjust  our  network.  The  provider  in  Melbourne  accidentally  pulled 
one  of  our  stubs  and  then  promptly  sent  all  of  their  personnel  to  assist  in  hurricane  recovery.  Four  days 
later  we  were  able  to  resume  testing,  resulting  in  an  eventual  extension  of  the  test  period  by  four  days. 

This  event  is  brought  up  not  because  it  was  fun,  Florida  during  a  hurricane  is  anything  but  fun,  but 
because  it  emphasizes  the  need  for  redundancy  and  spares  when  doing  distributed  testing.  Based  on  the 
forty  plus  days  of  rehearsal,  risk  reduction  tests,  and  integration  testing,  we  had  spare  components  at 
various  sites  to  handle  anticipated  breakdowns.  Unfortunately,  we  did  not  anticipate  Murphy.  By  the 
time  we  had  figured  out  how  to  reconfigure  our  network,  and  received  permission  to  do  so  from  the 
security  gods,  the  stub  was  replaced  and  our  original  network  was  back  up. 

What  this  means  to  the  distributed  tester,  is  that  during  meetings  of  the  entire  distributed  test  force,  he 
must  have  a  Murphy  meeting.  At  this  meeting  the  wildest  events  imaginable  must  be  listed  and  then  a 
course  of  action  evaluated  and  decided  upon.  For  many  events,  such  as  Florida  sinking  beneath  the  sea, 
there  will  no  work  around.  For  others,  such  as  the  event  just  described,  work  arounds  exist  and  can  at 
least  be  planned  for  and  preapproved. 

The  time  used  in  the  SE  must  be  synchronized  throughout  the  distributed  test  nodes.  The  ETE  test  used  a 
time  server  located  at  the  test  control  and  analysis  center  (TCAC)  that  worked  very  well,  and  ensured  that 
all  nodes  were  kept  to  within  one  millisecond  of  each  other.  Having  the  same  time  at  each  test  node 
helped  speed  up  the  analysis,  and  allowed  for  the  development  of  automated  tools  to  aid  in  the  analysis. 


Software  must  be  acquired  or  written  that  automatically  notifies  the  tester  in  a  timely  manner  when  there 
is  a  network  problem.  Since  most  software  of  this  type  is  intrusive  in  nature,  a  balance  must  be  arrived  at 
between  how  often  the  network  is  checked  and  the  network  load  due  to  the  test.  Adequate  bandwidth  must 
be  available  for  both  needs.  In  addition,  the  alarms  must  be  audible  alarms  because  network  problems 
always  occur  when  you  are  to  busy  to  watch  the  monitor  (Murphy  knows  how  to  do  ADS  Testing). 

Use  a  stepped  buildup  approach,  where  each  successive  ADS  test  builds  on  the  success  of  earlier  tests.  A 
“test,  analyze,  fix  and  build,  test”  approach  as  each  node  of  the  SE  is  added,  in  combination  with  a 
structured,  independent  testing  of  the  network,  will  greatly  improve  the  chances  for  successful  ADS 
testing. 


Effective  data  management  is  critical.  ADS  tests  can  and  do  generate  enormous  amounts  of  data  at 
distributed  locations.  Without  careful  planning,  key  data  may  not  be  collected  and/or  transmitted  to  the 
analysis  center,  and  data  collected  at  the  network  nodes  may  not  be  in  a  useful  form  for  centralized 
analysis.  Prior  to  the  beginning  of  the  test,  a  comprehensive  data  management  plan  must  clearly  identify 
the  data  to  be  collected  at  each  network  node,  on-site  processing  of  the  data,  and  the  data  to  be  transmitted 
to  the  analysis  center.  The  data  management  plan  must  be  rehearsed  and  refined  during  the  integration 
tests,  risk  reduction  tests,  and  rehearsals  and  adequate  time  must  be  allotted  for  data  analysis  between 
ADS  test  events. 

The  test  director  for  a  distributed  test  must  have  a  centrally  located  test  control  site.  This  test  control  site 
is  not  necessarily  geographically  centrally  located.  Instead  it  must  be  electronically  centrally  located  so 
that  it  can  monitor  and  control  all  that  occurs  within  the  test.  The  central  control  facility  must  have  the 
display  and  communications  capabilities  to  know  total  system  health  in  real  time.  Total  system  health 
includes  not  only  timely  and  continuous  status  reporting  from  the  real,  virtual,  and  constructive  players, 
but  the  data  processing  and  collection  system,  and  the  system  synchronization  mechanism.  The  central 
control  facility  must  also  support  voice  communications,  conferencing,  and  a  variety  of  other  human-to- 
human  interfaces  necessary  for  adequate  test  control.  Within  the  ETE  Test,  the  JADS  TCAC  was  this 
central  control  facility. 

Conclusion 


Succinctly  put,  ADS  has  significant  utility  in  supporting  the  testing  of  C4ISR  systems.  It  enables  the 
tester  to  explore  areas  of  the  system’s  performance  envelope  that  are  unexplorable  by  conventional  means. 
It  may  or  may  not  save  money,  but  it  will  definitely  allow  the  tester  to  gain  the  maximum  benefit  from  live 
test  events.  It  is  usually  more  difficult  to  do  than  live  testing  because  it  combines  elements  of  both.  As 
evidenced  by  the  Simulation  Test  and  Evaluation  Process  (STEP),  it  is  the  wave  of  the  future  and  we  must 
learn  how  to  do  it  efficiently  and  effectively. 
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